{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a3ff9afc",
   "metadata": {},
   "source": [
    "# pickle\n",
    "二进制格式文件，即使没有压缩，通常占用最少的磁盘空间，通常是最快的保存和加载。最简单的方法是使用pickle，不过手动处理二进制数据应该生成最小的文件大小。"
   ]
  },
  {
   "cell_type": "raw",
   "id": "2c30352a",
   "metadata": {},
   "source": [
    "在Python中长久的保存字符串、列表、字典等数据，方便以后使用，而不是简单的放入内存中。这个时候Pickle模块就派上用场了，它可以将对象转换为一种可以传输或存储的格式。\n",
    "pickle模块实现了用于序列化和反序列化Python对象结构的二进制协议。 “Pickling”是将Python对象层次结构转换为字节流(序列化)的过程， “unpickling”是反向操作（反序列化），从而将字节流（来自二进制文件或类似字节的对象）转换回对象层次结构。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7184cfa3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('this is a string', 42, [1, 2, 3], None)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pickle  as pickle\n",
    "t1 = ('this is a string',42,[1,2,3],None)\n",
    "t1"
   ]
  },
  {
   "cell_type": "raw",
   "id": "654dd99c",
   "metadata": {},
   "source": [
    "pickle.dump()\n",
    "封装是一个将Python数据对象转化为字节流的过程，拆封是封装的逆操作，将字节文件或字节对象中的字节流转化为Python数据对象，不要从不收信任的数据源中拆封数据。可以封装和拆封几乎任何Python数据对象"
   ]
  },
  {
   "cell_type": "raw",
   "id": "c826b288",
   "metadata": {},
   "source": [
    "pickle.dump(obj, file, protocol=None,)\n",
    "必填参数obj表示将要封装的对象\n",
    "必填参数file表示obj要写入的文件对象，file必须以二进制可写模式打开，即“wb”\n",
    "可选参数protocol表示告知pickler使用的协议，支持的协议有0,1,2,3，默认的协议是添加在Python 3中的协议3"
   ]
  },
  {
   "cell_type": "raw",
   "id": "2194d513",
   "metadata": {},
   "source": [
    "pickle.dumps(obj)：以字节对象形式返回封装的对象，不需要写入文件中"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "caf10163",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'\\x80\\x04\\x95$\\x00\\x00\\x00\\x00\\x00\\x00\\x00(\\x8c\\x10this is a string\\x94K*]\\x94(K\\x01K\\x02K\\x03eNt\\x94.'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p1 = pickle.dumps(t1)\n",
    "p1"
   ]
  },
  {
   "cell_type": "raw",
   "id": "eab041b1",
   "metadata": {},
   "source": [
    "pickle.load(file,*,fix_imports=True, encoding=\"ASCII\", errors=\"strict\")\n",
    "\n",
    "必填参数file必须以二进制可读模式打开，即“rb”，其他都为可选参数"
   ]
  },
  {
   "cell_type": "raw",
   "id": "5cfba113",
   "metadata": {},
   "source": [
    "pickle.loads(bytes_object): 从字节对象中读取被封装的对象，并返回"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "fa5871af",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('this is a string', 42, [1, 2, 3], None)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t2 = pickle.loads(p1)\n",
    "t1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5e52d79b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'(X\\x10\\x00\\x00\\x00this is a stringq\\x00K*]q\\x01(K\\x01K\\x02K\\x03eNtq\\x02.'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p2 = pickle.dumps(t1,True)\n",
    "p2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1cd0d00d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('this is a string', 42, [1, 2, 3], None)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t3=pickle.loads(p2)\n",
    "t3"
   ]
  },
  {
   "cell_type": "raw",
   "id": "b5fbbfe7",
   "metadata": {},
   "source": [
    "以下是几个例子："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "74ec34e6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'a': 123, 'b': 'ads', 'c': [[1, 2], [3, 4]]}\n"
     ]
    }
   ],
   "source": [
    "#序列化和反序列化一个字典对象\n",
    "import numpy as np\n",
    "import pickle\n",
    "\n",
    "f = open('test', 'wb')\n",
    "data = {'a':123, 'b':'ads', 'c':[[1,2],[3,4]]}\n",
    "pickle.dump(data, f)\n",
    "f.close()\n",
    "\n",
    "f1 = open('test', 'rb')\n",
    "data1 = pickle.load(f1)\n",
    "print(data1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "ee06ea8f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'\\x80\\x04\\x95.\\x00\\x00\\x00\\x00\\x00\\x00\\x00}\\x94(\\x8c\\x00\\x94h\\x01\\x8c\\x03age\\x94K \\x8c\\x04func\\x94\\x8c\\x08__main__\\x94\\x8c\\x05sayhi\\x94\\x93\\x94u.'\n"
     ]
    }
   ],
   "source": [
    "#序列化一个函数对象\n",
    "import pickle\n",
    "test = r'test.txt'\n",
    "#反序列化代码中也要定义相同的函数名称，函数体没限制\n",
    "def sayhi(name):\n",
    "    print(\"hello\",name)\n",
    "\n",
    "info = {\n",
    "    '':'',\n",
    "    'age':32,\n",
    "    'func':sayhi\n",
    "}\n",
    "\n",
    "print(pickle.dumps(info))\n",
    "\n",
    "with open(test,'wb') as f:\n",
    "    #f.write( pickle.dumps(info) )\n",
    "    pickle.dump(info,f)  #跟上面的f.write( pickle.dumps(info) )语意完全一样。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "e1ca3e60",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data>>> {'': '', 'age': 32, 'func': <function sayhi at 0x0000014E064BB670>}\n",
      "hello Alex\n",
      "hello2 Alex\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "#反序列化函数对象\n",
    "import pickle\n",
    "test = r'test.txt'\n",
    "#需要定义序列化代码中同样的函数名，函数体没限制\n",
    "def sayhi(name):\n",
    "    print(\"hello\",name)\n",
    "    print(\"hello2\",name)\n",
    "\n",
    "with open(test,'rb') as f:\n",
    "    # data = pickle.loads(f.read())\n",
    "    data = pickle.load(f)  #跟上面的data = pickle.loads(f.read())语意完全一样。\n",
    "    print('data>>>',data)\n",
    "\n",
    "print(data['func'](\"Alex\"))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "4beb7ee5",
   "metadata": {},
   "outputs": [],
   "source": [
    "#使用pickle将字典数据对象保存到文件\n",
    "import pickle\n",
    "\n",
    "data1 = {'a': [1, 2.0, 3, 4+6j],\n",
    "         'b': ('string', u'Unicode string'),\n",
    "         'c': None}\n",
    "\n",
    "selfref_list = [1, 2, 3]\n",
    "selfref_list.append(selfref_list)\n",
    "\n",
    "output = open('data.pkl', 'wb')\n",
    "\n",
    "# Pickle dictionary using protocol 0.\n",
    "pickle.dump(data1, output)\n",
    "\n",
    "# Pickle the list using the highest protocol available.\n",
    "pickle.dump(selfref_list, output, -1)\n",
    "\n",
    "output.close()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "9938dbb3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'a': [1, 2.0, 3, (4+6j)], 'b': ('string', 'Unicode string'), 'c': None}\n",
      "[1, 2, 3, <Recursion on list with id=1434619560192>]\n"
     ]
    }
   ],
   "source": [
    "#使用pickle从文件中重构Python对象\n",
    "import pprint, pickle\n",
    "\n",
    "pkl_file = open('data.pkl', 'rb')\n",
    "\n",
    "data1 = pickle.load(pkl_file)\n",
    "pprint.pprint(data1)\n",
    "\n",
    "data2 = pickle.load(pkl_file)\n",
    "pprint.pprint(data2)\n",
    "\n",
    "pkl_file.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4a5dc36",
   "metadata": {},
   "source": [
    "# struct \n",
    "struct模块包括用于在字节串和 Python 数据类型（如数字和字符串）之间进行转换的函数。"
   ]
  },
  {
   "cell_type": "raw",
   "id": "1ac1ede6",
   "metadata": {},
   "source": [
    "Packing（打包）和Unpacking（解包）\n",
    "Struct支持将数据packing(打包)成字符串，并能从字符串中逆向unpacking(解压)出数据。"
   ]
  },
  {
   "cell_type": "raw",
   "id": "96ecd311",
   "metadata": {},
   "source": [
    "struct.pack(fmt,v1,v2,.....)\n",
    "返回一个字节对象，该对象包含根据格式字符串格式打包的值v1、v2，…。参数必须与格式要求的值完全匹配。\n",
    "第一个参数是后面所有参数的格式字符串，后面的每个参数必须与格式字符串描述一致。\n",
    "eg:h表示short，l表示long； 'hhl'表示后面有三个参数，依次是short，short，long类型\n",
    "c表示 char，bytes of length 1（长度的byte数组），i表示integer 整数；'ci'表示后面有两个个参数，依次是char，integer类型"
   ]
  },
  {
   "cell_type": "raw",
   "id": "01d693b4",
   "metadata": {},
   "source": [
    "struct.unpack(fmt,string)\n",
    "返回一个由解包数据(string)得到的一个元组(tuple)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "4a8ecc6e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'\\x01\\x00\\x00\\x00\\x02\\x00\\x03'\n",
      "(1, 2, 3)\n"
     ]
    }
   ],
   "source": [
    "import struct\n",
    "# native byteorder\n",
    "buffer = struct.pack('ihb',1,2,3)\n",
    "print(buffer)\n",
    "s = struct.unpack('ihb',buffer)\n",
    "print(s)\n",
    "#首先将参数1,2,3打包，\n",
    "#打包前1,2,3明显属于python数据类型中的integer\n",
    "#pack后就变成了C结构的二进制串\n",
    "#转成 python的string类型来显是：'\\x01\\x00\\x00\\x00\\x02\\x00\\x03'\n",
    "#i 代表C struct中的int类型，故而本机占4位，1则表示为01000000\n",
    "#h 代表C struct中的short类型，占2位，故表示为0200\n",
    "#同理b 代表C struct中的signed char类型，占1位，故而表示为03"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "353ea71a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 2, 3)"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unpack('ihb',b'\\x01\\x00\\x00\\x00\\x02\\x00\\x03')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "b78dcadd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'\\x01\\x00\\x02\\x00\\x03\\x00\\x00\\x00'"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import struct\n",
    "pack('hhl', 1, 2, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "f36ac598",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 2, 3)"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "unpack('hhl',b'\\x01\\x00\\x02\\x00\\x03\\x00\\x00\\x00')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
